I. Field of the Invention
The present invention relates generally to image correction methods and, more specifically, to a process for adaptively removing green channel odd-even mismatch.
II. Background
As the sensor pixel count increases, the area of each pixel photodiode shrinks. The signal readout circuit has to take care of reading and transferring the weaker signal levels. For sensors with a RGB bayer pattern, the Green channel on the odd and even rows normally are read out via a different circuit. More specifically, the metal wire layout of the photo diode, electronic leakage, light incident angle and the signal output circuit, causes the green channel of a bayer pattern sensor to exhibit an unbalanced response. This imbalance contains both global and local variation. Although the circuit layout is identical, the imperfect manufacturing process can cause the read-out and amplifier circuit to be mismatched. Also, the non-uniformity of the color filter array and lens coating and mounting, etc., can also cause the green channel to exhibit odd-even mismatch. Therefore, the overall green channel odd-even mismatch is location dependent and non-uniform. The green channel odd-even mismatch makes the image processing task difficult because the green channel odd-even mismatch translates into cross hatched patterns of artifact as shown in FIG. 1.
In FIG. 1, a flat field image 10 was created with demosaic operation. This flat field image is supposed to be flat because the lens was covered with a diffusion lens. There should not be any texture on the image after it is processed. However, as is seen in FIG. 1, cross hatched patterns are prevalent across the entire image 10. Further investigation reveals that this artifact is caused by the green channel odd-even mismatch.
The demosaic algorithm normally depends greatly on the green channel signal to determine the edge because 50% of the bayer pixels are green. An exemplary bayer pixel arrangement is shown in FIG. 10B. However, if there is a green channel odd-even mismatch, such mismatch is treated as an edge and the demosaic module tries to preserve such edge in either vertical or horizontal directions. The end result is the cross hatched patterns shown in FIG. 1 after demosaic processing. This artifact is most obvious when the image is zoomed in around 300%.
One failed solution proposed a global green channel gain balance. If the channel read-out and amplifier circuit were the only factors for green odd-even mismatch, then applying a global green channel gain balance may solve the problem. However, for a Sony™ 3 MP sensor, the use of a global green channel gain balance did not work. Further analysis reveals that the odd-even mismatch is not uniform across the entire image.
Dividing the 3 MP sensor image into regions with 32×32 pixels per region, the flat field image is performed with a region-based channel balance calibration. The required Gr gain and Gb gain to balance the green channel is shown in the FIGS. 2A and 2B. As can be easily seen from FIGS. 2A and 2B, the green channel balance is very non-uniform across the entire image. As a result, applying global green channel gains can not solve the problem or eliminate the cross hatched pattern of artifact shown in FIG. 1.
Another possible solution employs an adaptive bayer filter. The adaptive bayer filter can be applied only on green pixels to smooth out the odd-even mismatch. The issue is, for the Sony sensor under study, some regions show a green channel odd-even mismatch of 13%. If such a large mismatch is intended to be smoothed out, the true edges in the images may suffer too. As a result, the images will be blurred.
Furthermore, the computation cost of the adaptive bayer filter is relatively high in terms of software/firmware. The computations would also add a considerable amount of delay time to the snap shot image processing. FIG. 3 illustrates the resultant image 20 after applying an adaptive bayer filter to the flat field image of FIG. 1. The resultant image 20 has gone through the entire processing pipeline. A moderate amount of smoothing is applied in the adaptive bayer filter. While, in the resultant image 20 some cross hatched pattern artifact is smoothed out, some still remains.
If a much larger amount of smoothing is applied in the adaptive bayer filter, the cross hatched patterns can be completely removed but at the cost of blurred texture in the images.
If a straightforward smoothing is performed on the raw images on the bayer domain, the edges and textures will suffer. If each pair of green pixels (Gr and Gb) is forced to be equal, the high frequency edges suffer.